A Process for Evaluating a Software Bidding Model

This report discusses the issues involved in evaluating a software bidding model. We found it difficult to assess the appropriateness of any model evaluation activities without a baseline or standard against which to assess them. This paper describes our attempt to construct such a baseline. We reviewed evaluation criteria used to assess cost models and an evaluation framework that was intended to assess the quality of requirements models. We developed an extended evaluation framework that will be used to evaluating our bidding model. Furthermore, we suggest the evaluation framework might be suitable for evaluating other models derived from expert opinion based influence diagrams. We use a simple process model to relate the evaluation framework to the model building process. The process model indicates the order in which different evaluation activities are performed and the role responsible for performing them. It also illustrates the difference between evaluating a generic bidding model and evaluating a specialized bidding model.